Understanding Coverbal Dimensional Gestures in a Virtual Design Environment
نویسنده
چکیده
Todays multimodal systems, which allow full-body (3D) gestures and speech as input modalities, are quite restricted to easily interpretable coverbal gestures with a prede ned shape and meaning. In this paper, we propose methods to abstract the concrete shape of gestures by using high-level features and to integrate them with coexpressive words using their phonological attributes. The application of this approach is discussed for a class of gestures useful in virtual design. We sketch our technical environment and rst implementation approaches to build a prototype system.
منابع مشابه
Coverbal Iconic Gestures for Object Descriptions in Virtual Environments: An Empirical Study
متن کامل
Processing Iconic Gestures in a Multimodal Virtual Construction Environment
In this paper we describe how coverbal iconic gestures can be used to express shape-related references to objects in a Virtual Construction Environment. Shape information is represented using Imagistic Description Trees (IDTs), an extended semantic representation which includes relational information (as well as numerical data) about the objects’ spatial features. The IDTs are generated online ...
متن کاملModel-based Animation of Coverbal Gesture
Virtual conversational agents are supposed to combine speech with nonverbal modalities for intelligible and believeable utterances. However, the automatic synthesis of coverbal gestures still struggles with several problems like naturalness in procedurally generated animations, flexibility in pre-defined movements, and synchronization with speech. In this paper, we focus on generating complex m...
متن کاملA Model for the Representation and Processing of Shape in Coverbal Iconic Gestures
Verbal descriptions of object shape are frequently accompanied by iconic gestures expressing meaning via similarity to the referent. This paper presents a study examining the morphological variety of shape-related iconic gestures and the way they express aspects of object shape. It is shown that information about object extent, object boundary, and part structure is reflected both in coverbal g...
متن کاملSpherical Mechanism Synthesis in Virtual Reality
This paper presents a new approach to using virtual reality (VR) to design spherical mechanisms. VR provides a three-dimensional (3-D) design space where a designer can input design positions using a combination of hand gestures and motions and view the resultant mechanism in stereo using natural head movement to change the viewpoint. Because of the three-dimensional nature of the design and ve...
متن کامل